A norm-concentration argument for non-convex regularisation

However, independent results in several areas indicate added value to non-convex norm regularisation, despite the existence of local optima. Work in statistics (Fan & Li, 2001) and signal reconstruction (Wipf & Rao, 2005) have established the oracle properties of non-convex regularisers. Good empirical results were also reported in signal processing (Chartland, 2007) and SVM classification (Weston et.al, 2003). Furthermore, using a family of non-convex norms that we shall refer to as fractional-norms in the rest of the paper, turned out to consistently outperform the L1 regulariser in real high-dimensional genomic data classification (Liu et.al, 2007), both in terms of error rates and interpretability. Related ideas, termed as ’zeronorm’ regularisation (Weston et.al, 2003) were also found useful in many other applications, though their success appeared to be data dependent. It is therefore of interest to gain a better understanding of the potential advantages of non-convex norm regularisers, which is our purpose.